Skip to content

fix(rivetkit-core): cancel driver alarm before sqlite teardown on destroy - #5530

Open
abcxff wants to merge 1 commit into
stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqprofrom
container-runner-fixes
Open

fix(rivetkit-core): cancel driver alarm before sqlite teardown on destroy#5530
abcxff wants to merge 1 commit into
stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqprofrom
container-runner-fixes

Conversation

@abcxff

@abcxff abcxff commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@claude

claude Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review

Re-verified after rebase (commit c0744a7, same diff as prior review).

Small, well-scoped fix. The reordering is correct and fixes a real race.

What the change does

Moves ctx.cancel_driver_alarm_logged() (Destroy-only) to run right after sync_alarm_logged() and before wait_for_pending_alarm_writes() / cleanup_sqlite(), instead of after SQLite teardown. cancel_driver_alarm_logged calls set_alarm_tracked(None), which spawns a task that persists last_pushed_alarm to SQLite once the envoy ack arrives (persist_last_pushed_alarm in actor/state.rs:593, backed by internal_storage::persist_last_pushed_alarm). With the old ordering, that write ran after cleanup_for_shutdown had already closed the local SQLite backend (sqlite/mod.rs::close_backend), so it could fail against a closed connection (surfacing as sqlite.transaction_closed) and was never awaited by shutdown anyway. Moving the call earlier means wait_for_pending_alarm_writes() now drains and awaits that specific pending write before SQLite is torn down, closing the race.

This also brings Destroy in line with the existing pattern already used for sync_alarm_logged()'s pending write (already awaited via wait_for_pending_alarm_writes() before cleanup), so it's consistent with precedent rather than a new architecture. The match reason { Destroy => ..., Sleep => {} } arms are exhaustive per repo convention (no _ fallthrough), and the untouched Sleep path (cancel_local_alarm_timeouts() after cleanup, keeping the persisted engine alarm armed for the next instance) still looks correct.

Minor notes (non-blocking)

  • Test coverage: destroy_shutdown_still_clears_driver_alarm_after_cleanup and sleep_shutdown_preserves_driver_alarm_after_cleanup in tests/task.rs only assert test_driver_alarm_cancel_count(), which is unchanged by this reordering. Nothing in the diff would fail if the old (buggy) ordering were reintroduced. Consider a regression test that specifically exercises "the pending alarm-cancel write completes before SQLite cleanup starts" (or fault-injects to reproduce the old sqlite.transaction_closed), so the fix can't silently regress. Note the destroy test's name (..._after_cleanup) is now slightly stale since the cancel happens before cleanup, worth a rename in the same pass if a new test is added.
  • New blocking dependency on shutdown path: awaiting the pending alarm write means Destroy shutdown now blocks on the envoy round-trip ack for set_alarm(None) before SQLite cleanup proceeds. This mirrors existing behavior for the sync_alarm_logged write, so it isn't a new risk class, but worth confirming there's an overall shutdown deadline/timeout upstream so a slow or unresponsive envoy can't stall actor destroy indefinitely.
  • Comment style: the added comment includes a parenthetical ((rather than after cleanup_sqlite)) that the repo's CLAUDE.md style guidance discourages ("avoid fragmented structures with parentheticals and dashes"). Minor, but could be rephrased as its own sentence. Also, it explains both the Destroy-path reordering here and the Sleep-path behavior further down (cancel_local_alarm_timeouts after cleanup); consider moving the Sleep explanation to a short comment at that call site so each match arm's reasoning stays local.

No security or naming/style concerns beyond the above. The fix is narrowly targeted and doesn't touch unrelated code paths.

@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from d84e30f to b3a2e19 Compare July 31, 2026 05:27
@abcxff
abcxff force-pushed the container-runner-fixes branch from 1a48904 to 858d26f Compare July 31, 2026 05:27
@abcxff
abcxff force-pushed the container-runner-fixes branch from 858d26f to 84962e8 Compare July 31, 2026 06:03
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from b3a2e19 to 3e9dbf4 Compare July 31, 2026 06:03
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from 3e9dbf4 to ad0f63f Compare August 3, 2026 20:17
@abcxff
abcxff force-pushed the container-runner-fixes branch from 84962e8 to 58cfe9e Compare August 3, 2026 20:17
@abcxff
abcxff force-pushed the container-runner-fixes branch from 58cfe9e to 486fd99 Compare August 10, 2026 17:16
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from ad0f63f to 2fd10c6 Compare August 10, 2026 17:16
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from 2fd10c6 to cc14c96 Compare August 10, 2026 19:03
@abcxff
abcxff force-pushed the container-runner-fixes branch from 486fd99 to 51bb31e Compare August 10, 2026 19:03
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from cc14c96 to d902009 Compare August 10, 2026 20:35
@abcxff
abcxff force-pushed the container-runner-fixes branch from 51bb31e to 1cf1038 Compare August 10, 2026 20:35
@abcxff
abcxff force-pushed the container-runner-fixes branch from 1cf1038 to c0744a7 Compare August 10, 2026 22:09
@abcxff
abcxff force-pushed the stack/feat-container-runner-keep-instance-warm-instead-of-self-exiting-qqzrqpro branch from 7562a64 to ed6f3b4 Compare August 11, 2026 13:39
@abcxff
abcxff force-pushed the container-runner-fixes branch from c0744a7 to 2f59725 Compare August 11, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant